-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[REFACTOR] store webhook event in database #29145
Conversation
Co-authored-by: silverwind <[email protected]>
@jolheiser do you know if/when you will have some time to look at this PR? (especially since it should reduce the changes needed for #19307) |
@silverwind @KN4CK3R thank you for your initial comments. Do you have any other feedback? Do you think that there is a chance for this PR to make it before 1.22 ? (cc @delvh) |
If we merge it this week, then yes. |
I'm not well versed around these parts but what is the deal with this |
Could you have more description on the issue body about the plan? Why you add the new version column and how will the new version support customized webhooks in the future? |
@silverwind & @lunny I haved updated my initial description to emphasize:
Does it addresses your concerns? |
What I haven't understood yet: |
The version 1 is set to the hooktask which were generated before the upgrade (set to So the "new" codebase should only create version |
I think we are talking about different things: I expected to see the version 2 being set somewhere (apart from the tests).
Yes, that's exactly what I thought. |
@delvh it is done upon hooktask creation in services/webhook/webhook.go:172 webhook_model.CreateHookTask(ctx, &webhook_model.HookTask{
HookID: w.ID,
PayloadContent: string(payload),
EventType: event,
PayloadVersion: 2,
}) |
Now it makes sense why I don't find anything when I'm searching for |
Co-authored-by: delvh <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall looks good from initial look. I'll try and take some time to toy around with it more.
Co-authored-by: John Olheiser <[email protected]>
As soon as there is willingness for this PR to be merged, I can solve the merge conflicts. |
I guess that would be now |
I brought this up on the TOC call, and how this PR would be great to get in, Im just wondering if it'll still include a raw copy of the body and headers that are sent to the endpoints? As having that information has been helpful to be able to copy/paste it into a curl to debug issues. (Not blocking getting this in, I'm just asking to make sure no existing functionality is removed by this PR) |
Yes, that part appears to be largely unchanged https://github.com/go-gitea/gitea/pull/29145/files#diff-2402142c5412a93d0288dee23e4fa6d8d5b1e8a07b627714820a2344ff205125R161 |
Yes it does, a little bit adapted, because the body must be saved as well (since the payloadBody is now the raw event): // Record delivery information.
t.RequestInfo = &webhook_model.HookRequest{
URL: req.URL.String(),
HTTPMethod: req.Method,
Headers: map[string]string{},
Body: string(body),
}
for k, vals := range req.Header {
t.RequestInfo.Headers[k] = strings.Join(vals, ",")
} I have solved the merge conflicts. |
* giteaofficial/main: Store webhook event in database (go-gitea#29145) Fix bug hidden on CI and make ci failed if tests failure (go-gitea#29254)
This reverts commit 26653b1.
This reverts commit ec30295.
Refactor the webhook logic, to have the type-dependent processing happen only in one place.
Current webhook flow
This means that webhook-type dependant logic is needed in step 2 and 3. This is cumbersome and brittle to maintain.
Updated webhook flow with this PR:
So the only webhook-type dependent logic happens in one place (step 3) which should be much more robust.
Consequences of the refactor
payload_version
is added (version 1: the body has already been pre-process / version 2: the body is the raw event)So future webhook additions will only have to deal with creating an http.Request based on the raw event (no need to adjust the code in multiple places, like currently).
Moreover since this processing happens when fetching from the task queue, it ensures that the queuing of new events (upon a
git push
for instance) does not get slowed down by a slow webhook.As a concrete example, the PR #19307 for custom webhooks, should be substantially smaller:
services/webhook/deliver.go
services/webhook/webhook.go
(add the new webhook to the map)*webhook_model.Webhook
is provided as argument)